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1. INTRODUCTION 

Occasionally, the space shuttle must land at 
Edwards Air Force Base (EAFB) in Southern 
California when weather conditions at Kennedy 
Space Center (KSC) violate Flight Rules (FR). 
However, the complex terrain in and around EAFB 
makes forecasting surface winds a challenge for 
the Spaceflight Meteorology Group (SMG). In 
particular, “wind cycling" cases, in which the wind 
speeds and directions oscillate among towers near 
the EAFB runway, present a challenging forecast 
problem for shuttle landings. An accurate depiction 
of the winds along the runway is crucial in making 
the landing decision. Global and national scale 
numerical weather prediction models cannot 
properly resolve the wind field due to their coarse 
horizontal resolutions, so a properly tuned high- 
resolution mesoscale model is needed. The 
Weather Research and Forecasting (WRF) model 
meets this requirement. 

The goal of this work was to assess different 
WRF model options and to determine how well the 
model could predict surface wind speed and 
direction at EAFB and if one model configuration 
performed better than others. The data presented 
in this report shows many interesting phenomena 
that were not investigated in the interest of 
focusing on the SMG goal of model capability and 
time constraints to complete the work. Given the 
time constraints and the observation that both 
wind speed and direction indicated cycling during 
an event, The Applied Meteorology Unit (AMU) 
chose to focus on assessing model performance 
forecasting wind speed. 

The AMU investigated WRF’s two dynamical 
cores: 1) the Advanced Research WRF (ARW) 
and 2) the Non-hydrostatic Mesoscale Model 
(NMM). Besides the two dynamical cores, the 
AMU also considered two options for a “hot-start” 
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initialization of the WRF model: 1) the Local 
Analysis and Prediction System (LAPS; McGinley 
1995) and 2) the Advanced Regional Prediction 
System (ARPS) Data Analysis System (ADAS; 
Brewster 1996). Both LAPS and ADAS are three- 
dimensional weather analysis systems that 
integrate multiple meteorological data sources into 
one analysis over the user’s domain of interest. 
These analysis systems allow mesoscale models 
to benefit from the addition of high-resolution data 
sources in their initial conditions. In addition to 
model core and initialization options, there are 
many different model parameterization options 
within each core. Having a series of initialization 
options and WRF cores, as well as many options 
within each core, provides SMG with considerable 
flexibility as well as decision-making challenges. 
The goal of this study is to assess the different 
model configurations and to determine which 
configuration will best predict surface wind speed 
and direction at EAFB. To accomplish this, the 
AMU compared the WRF model performance 
using ADAS versus LAPS for the ARW and NMM 
cores and compared model performance while 
varying the physics options. 

2. BACKGROUND 

The location of EAFB (Figure 1) is on the 
western edge of the Mojave Desert. It is 
surrounded by mountains on three sides: the San 
Bernardino Mountains to the southwest, the San 
Gabriel Mountains to the south, and the Tehachapi 
Mountains and southern Sierra Nevada to the 
west and northwest. The base is located next to 
Rogers Dry Lake whose hard surface acts as an 
extension to the EAFB runways. There are twelve 
wind towers located in and around the EAFB 
runways. Figure 2 shows the locations of the 
towers along the EAFB runway complex. 

The complex terrain located in and around 
EAFB creates the erratic behavior of the wind 
speed and direction seen on the EAFB wind 
towers. Wind cycling events occur when there is 
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Figure 1. Location of EAFB in relation to surrounding mountainous terrain. Background image from MSN Live 
Search Maps. 



Figure 2. Wind tower locations on EAFB. The towers along the concrete and lakebed runways are indicated by 
arrows. Background image from MSN Live Search Maps. 



an oscillation in wind direction and/or wind speed 
among the wind towers in the network near the 
EAFB runway complex. During these cycling 
events, the wind speed and direction reported 
from the towers near the concrete runway (Towers 
44, 220, 224) are noticeably different than that 
reported from towers near the lakebed runway 
(Towers 154, 230, 234). These events usually last 
from 90 minutes up to 4 hours or longer and most 
often occur when the prevailing wind is from the 
northwest or west-northwest. 

The wind cycling events at EAFB are thought 
to be caused by mountain waves created as winds 
from the northwest flow over the Tehachapi 
Mountains. Figure 3 shows several mountain wave 
clouds that developed near EAFB on 30 January 
2008. On this day, the wind along the concrete 
runway fluctuated from westerly and relatively 
lighter to northwesterly and stronger. The winds 
along the lakebed were northwesterly and strong 
during the wind cycling event. 


3. DATA AND METHODOLOGY 

The important aspects of this work were the 
choice of wind cycling case days, the model 
configuration, and the data used to initialize the 
models. The candidate wind cycling days chosen 
occurred between December 2006 and July 2008. 


This report presents the findings from a one- 
year study of model sensitivities for predicting 
wind cycling events at EAFB. This analysis 
examined different model cores, initializations, and 
physics options to determine the impact on model 
skill. Two local data analysis systems that 
ingested satellite data, radar data, and surface 
observations across southern California were used 
to initialize a mesoscale model. Model skill was 
assessed using both a subjective and objective 
verification that compared forecast to observed 
wind speed and direction from the local EAFB 
wind towers. Section 3 describes the data and 
methodology including an example of a wind 
cycling case at EAFB, the different model cores, 
model initialization options, and the data ingested. 
The results of the subjective and objective 
analyses are presented in Sections 4 and Section 
5, respectively. Section 6 summarizes the report 
and provides recommendations on the best 
mesoscale model configuration for operational use 
based on this study. 


Comparisons were made between the ARW and 
NMM core initialized with ADAS and LAPS, as well 
as with different physics parameterizations. Both 
subjective and objective analyses were used to 
verify the forecasts. 



Figure 3. Visible satellite image showing several mountain wave clouds on 30 January 2008 near EAFB. The yellow 
circle indicates the location of EAFB. 



3.1 Wind Cycling Candidate Days 

The period of record for choosing wind cycling 
candidate days was December 2006 through July 
2008. SMG provided the AMU with the dates of 
seven wind cycling cases. Six of the seven were 
strong wind cycling cases and one, the 30/31 July 
2008 case, was marginal in that the fluctuations in 
wind speed and direction were smaller than for all 
other wind cycling cases. Two null, or non-wind 
cycling, days were randomly chosen from the 
data. Table 1 lists the candidate days, the start of 
the wind cycling event, and the end of the event. 


Table 1. List of all wind cycling candidate 
days, null cases, start of the wind 
cycling event, and the end of the 
event. 

Candidate Day 

Start of Event 

End of Event 

22 Dec 2006 

0900 UTC 

1800 UTC 

30 Jan 2008 

1030 UTC 

1330 UTC 

14 Feb 2008 

0300 UTC 

0700 UTC 

5 Mar 2008 

0000 UTC 

1000 UTC 

4 Jun 2008 

1100 UTC 

2000 UTC 

7 Jun 2008 

0200 UTC 

1300 UTC 

30/31 Jul 2008 

1900 UTC 

0100 UTC 

9 Jun 2007 

NULL CASE 

1 Nov 2007 

NULL CASE 


Figure 4 shows a time series of wind speed 
and direction at Towers 44 and 234 from a wind 
cycling event that occurred on 14 February 2008. 
Inspection of Figure 4 reveals a wave-like 
behavior in the time series between 0300 and 
0700 UTC (denoted by the blue vertical lines in 
Figure 4) for wind speed and direction at Tower 44 
(top), which is located near the concrete runway. 
During this time there was an oscillation between 
southwesterly and northwesterly wind, as well as a 
5 to 15 kt change in wind speed, that occurred 
approximately every half hour. The wind at Tower 
234, near the lakebed runway, did not exhibit this 
wave-like behavior. 

3.2 Model Core and Initialization Options 

For this study, the AMU employed both the 
ARW and NMM cores of the WRF model. The 
ARW core was developed primarily at the National 
Center for Atmospheric Research (NCAR) and is a 
fully compressible, non-hydrostatic mesoscale 
model with a hydrostatic option. It consists of a 
mass-based hydrostatic pressure terrain following 
coordinate, Arakawa C-grid staggering for the 
horizontal grid, time-split integration using a third 
order Runge-Kutta scheme with a small step for 


acoustic and gravity wave modes, and up to sixth 
order advection options in the horizontal and 
vertical (Skamarock et al. 2005). There are also 
full physics options for microphysics, planetary 
boundary layer, cumulus parameterization, 
radiation, and land surface schemes (Skamarock 
et al. 2005). The NMM core is also a fully 
compressible, non-hydrostatic mesoscale model 
with a hydrostatic option (Janjic et al. 2001, Janjic 
2003a, b). It consists of a hybrid sigma-pressure, 
terrain following vertical coordinate, Arakawa E- 
grid, a forward-backward time integration scheme, 
a second order advection option in the horizontal 



Figure 4. Wind direction (degrees) and wind speed 
(kts) for Tower 44 (top) and Tower 234 (bottom) on 14 
February 2008 from 0000 to 12000 UTC. The green line 
is the wind direction, the red line is the peak wind and 
the blue line denotes the steady-state wind. Wind 
cycling occurs in the time period between the vertical 
dashed blue lines. 

enstrophy (Janjic 1984). Most physics packages 
available to the ARW have not been tested with 
the NMM and therefore, the physics options for the 
NMM are more limited than for the ARW. 

A “hot-start" initialization of the WRF model 
can be made using LAPS or ADAS. LAPS is a 
data assimilation tool that uses numerous 



meteorological observations, such as satellite 
data, radar data, and surface observations, to 
generate a three-dimensional representation of the 
atmospheric forcing fields, such as wind speed 
and direction, surface temperature and pressure, 
relative humidity, precipitation and cloud cover 
(McGinley et al. 1991; Albers 1995; Albers et al. 
1996; Birkenheuer 1999; McGinley 1995). It 
includes a wind analysis and a three-dimensional 
cloud analysis, which are needed for the LAPS 
hot-start initialization. The LAPS cloud analysis is 
designed to create consistency with all data and 
the typical meteorology of clouds by combining 
data from infrared and visible satellite data, three- 
dimensional LAPS radar reflectivity derived from 
the full-volume radar data, and the LAPS three- 
dimensional temperatures (Albers et al. 1996). 
Fields derived from the cloud analysis include 
cloud liquid water, cloud type, cloud droplet size, 
and icing severity (Albers et al. 1996). However, in 
the latest versions of LAPS, the option to initialize 
precipitation was turned off. Mr. Chris Anderson of 
the National Oceanic and Atmospheric 
Administration Earth System Research Laboratory 
Global Systems Division (GSD) indicated that at a 
1-km grid spacing, initializing WRF with 
hydrometeors could create a large outflow that 
would degrade the forecasts. Consequently, GSD 
zeroed out the precipitation particles in the 
initialization, but left the cloud particles. He 
recommended keeping the precipitation 
initialization turned off if it was not an essential 
component of the research. Therefore, the AMU 
decided not to initialize precipitation since this 
study was not focused on convective forecasting. 


The ADAS system, developed by the 
University of Oklahoma, has two main 
components. The first is a Bratseth objective 
analysis scheme that evaluates pressure, wind, 
potential temperature, and specific humidity. The 
second component is a three-dimensional cloud 
analysis scheme that is used for the hot-start 
initialization (Zhang et al. 1998). The ADAS cloud 
analysis is based on the LAPS cloud analysis with 
some modifications (Case et al. 2002). It uses 
surface observations of cloud cover and height, 
satellite data, and radar data to determine the 
cloud cover, cloud liquid and ice water, cloud type, 
rain/snow/hail mixing ratios, icing severity, in-cloud 
vertical velocity, cloud base and top, and cloud 
ceiling (Case et al. 2002; Zhang et al. 1998; 
Brewster 2002). 

3.3 Data Ingest 

Data ingested by the model through either the 
LAPS or ADAS analysis packages included Level 
II Weather Surveillance Radar-1988 Doppler 
(WSR-88D) data from the Las Vegas, NV (KESX), 
Yuma, AZ (KYUX), Santa Ana Mountains, CA 
(KSOX), San Joaquin Valley, CA (KHNX), San 
Diego, CA (KNKX), and Los Angeles, CA (KVTX) 
radars (Figure 5), Geostationary Operational 
Environmental Satellites (GOES) visible and 
infrared satellite imagery, Meteorological 

Assimilation Data Ingest System (MADIS; 
http://madis. noaa.gov/ ) data, and local EAFB wind 
tower data. The Level II WSR-88D data contained 
full volume scans of reflectivity at a resolution of 1 ° 
by 1 km, radial velocity at 1° by 0.25 km, and 
spectrum width data at a 1° by 0.25 km (Fulton et 



Figure 5. Locations of WSR-88D sites (yellow) used in creating the high-resolution LAPS and 
ADAS analyses. The black circle shows the location of EAFB. 


al. 1998). These data were available every 4 to 6 
minutes. The GOES-12 visible imagery was 
available at a 4 km horizontal resolution also every 
15 minutes. Both visible and infrared imagery 
provided brightness temperatures to the analysis 
packages. Each initialization analysis used a 
different background model to create the first- 
guess background fields. The choice of 
background data for both LAPS and ADAS was 
based on the highest resolution model data 
currently supported in the latest versions of both 
analysis packages. The latest version of LAPS 
allows the use of the Rapid Update Cycle (RUC) 
20-km model as background data, while ADAS 
supports the North American Model (NAM) 12-km 
data. 

A suite of scripts written by the AMU formed 
the core for initializing the WRF model with ADAS 
output. The analysis package ingested satellite 
data in Man computer Interactive Data Access 
System (MclDAS) Area format 15 minutes prior to 
model initialization time; raw, full volume radar 
data within 10 minutes of the model initialization 
time; and surface data from the hour of and hour 
prior to model initialization. The largest task in 
configuring LAPS was working with the ingest 
code. The ingest code can only be used with raw 
data that have the same configuration and format 
as GSD’s raw data, which is the Network Common 
Data Form (NetCDF). Therefore, all data files were 
converted to NetCDF format to be used within 
LAPS. Software to convert the WSR-88D data to 
NetCDF format was obtained from GSD. Scripts 
were written to convert raw wind tower data into a 
format ingestible by both LAPS and ADAS. 
Converting satellite data in MclDAS Area format to 
NetCDF required several steps. The data were 
first ported to the local Meteorological Interactive 
Data Display System (MIDDS) system where they 
were remapped to the Lambert Conformal 
projection using the IMGREMAP command. Next, 
the remapped data were run through a program 
that converted them to NetCDF format. This 
program is called AreaToNetCDF and is available 
from the Space Science and Engineering Center 
(SSEC) at the University of Wisconsin. All of the 
reformatted data files were then ingested into 
LAPS to create an initialization field for the model. 

3.4 Model Configuration 

Each model simulation was run at a 1-km 
horizontal grid spacing over EAFB and adjacent 
mountainous areas with 51 irregularly spaced, 
vertical sigma levels. Each run was integrated 12 
hours and began one to three hours before the 


onset of the observed wind cycling event. Table 2 
lists the WRF model run times for each of the 
candidate days. Several factors were considered 
in determining the grid spacing of the WRF runs: 
the average distance between the wind towers, 
the customer requirements, and the computational 
requirements to run a high-resolution model. 
Grasso (2000) notes the theoretical minimum 
resolvable wavelength by a model is twice the 
horizontal grid spacing, but Walters (2000) 
suggests that as many as 10 grid points may be 
required for a more reasonable representation of 
the true solution. The AMU determined a 2-km or 
less horizontal grid spacing would resolve the wind 
cycling events given the average distance of 8 km 
between wind towers on the runway (44, 220, and 
224) and the lakebed (154, 230, and 234) and 
taking into account the spacing of all the wind 
towers. Customer requirements indicated a 1.3-km 
or less grid spacing was desired. To determine the 
highest resolution that could be run in real-time, 
three different runs using 1.33, 1, and 0.66-km grid 
spacing were made to determine the optimal 
resolution for the WRF forecasts. The 1-km run 
slightly outperformed the 1.3-km run with respect 
to wind speed and direction forecasts. The model 
was then run with a 0.66-km grid spacing, but it 
took more than twice as long to run as compared 
to the 1-km WRF forecast. This is not practical for 
real-time operations, so the 1-km grid spacing was 
used. 


Table 2. List of WRF model run times for all wind 

cycling and null cases. 


Candidate 

Model Start 

Model End 

Day 

Time 

Time 

22 Dec 2006 

0600 UTC 

1800 UTC 

30 Jan 2008 

0900 UTC 

2100 UTC 

14 Feb 2008 

0000 UTC 

1200 UTC 

4/5 Mar 2008 

2100 UTC 

0900 UTC 

4 Jun 2008 

0900 UTC 

2100 UTC 

7 Jun 2008 

0000 UTC 

1200 UTC 

30/31 Jul 2008 

1800 UTC 

0600 UTC 

9 Jun 2007 

0600 UTC 

1800 UTC 

1 Nov 2007 

0900 UTC 

2100 UTC 


Boundary conditions for the WRF runs were 
obtained from the NAM model with a horizontal 
grid spacing of 12-km. The NAM model domain is 
on a Lambert conformal projection with a 
horizontal domain size of 614 by 239 grid points 
and 42 vertical levels. It produces an 84-hr 
forecast every six hours, or four times per day. 
This model was chosen for boundary conditions as 
it has the best resolution of all available datasets. 



Each candidate day consisted of six runs 
using different model configurations, for a total of 
42 runs for the wind cycling case days and 14 for 
the null cases. The different configurations 
included: 

• LAPS-ARW with the Yonsei University 
planetary boundary layer (PBL) scheme 
and NCAR/Penn State Mesoscale Model 
Version 5 (MM5) similarity surface layer 
scheme (LAPSARW_Yonsei), 

• LAPS-ARW with the Mellor-Yamada-Janjic 
PBL scheme and ETA similarity surface 
layer scheme (LAPSARW_MYJ), 

• LAPS-NMM with the National Center for 
Environmental Prediction (NCEP) Global 
Forecast Systems (GFS) PBL scheme and 
NCEP GFS surface layer scheme 
(LAPSNMM_GFS), 

• LAPS-NMM with the Mellor-Yamada-Janjic 
PBL scheme and ETA similarity surface 
layer scheme (LAPSNMM_MYJ), 

• ADAS-ARW with the Yonsei University PBL 
scheme and MM5 similarity surface layer 
scheme (ADASARW_Yonsei), and 

• ADAS-ARW with the Mellor-Yamada-Janjic 
PBL scheme and ETA similarity surface 
layer scheme (ADASARW_MYJ). 

All other physics parameters were the same 
for each ARW model run. The NMM runs did not 
use the same physics as the ARW runs since the 
ultimate goal of the task was to see if wind cycling 
could be simulated. Instead, the best and most 
widely tested choice of physics found in the 
literature for the NMM core were used. Table 3 
lists the physics options used in both the ARW and 
NMM runs that were held constant. 

4. SUBJECTIVE ANALYSIS 

The AMU completed a subjective analysis of 
the WRF forecasts for all candidate days. The 
goals of the subjective analysis were to determine 

• If the model was able to predict the timing 
and/or magnitude of the wind cycling 
events at the concrete runway towers by 
comparing the observed wind speed to the 
forecast wind speed and 

• If the model could provide the forecasters 
with an indication of whether or not a wind 
cycling event was likely to occur by 
assessing the model forecasts on wind 
cycling days and null case days. 


Table 3. List of the physics options used for each 
model run for both the ARW and NMM cores. 


ARW 

NMM 

Microphysical 

scheme 

Lin et al. 
(1983) 

Ferrier (1994) 

Cumulus 

Parameterization 

None 

None 

Land surface 
option 

Noah Land 
Surface 
Model (Chen 
and Dudhia 
2001) 

Noah Land 

Surface Model 

(Chen and 

Dudhia 2001) 

Shortwave 

radiation 

scheme 

Goddard 
(Chou and 
Suarez 
1994) 

GFDL (Laois and 
Hansen 1974) 

Longwave 

radiation 

scheme 

RRTM 

(Mlawer et 
al. 1997) 

GFDL (Fels and 

Schwarzkopf 

1975; 

Schwarzkopf and 
Fels 1985, 1991) 


4. 1 Model Performance at the Concrete 
Runway Towers 

Using the Grid Analysis and Display System 
software, the AMU extracted the model forecast 
winds for every model run at each grid point 
nearest to each of the three concrete runway 
towers (44, 224 and 220 as shown in Figure 2) for 
comparison. The three tower locations and 
nearest corresponding grid point used in the 
comparison are shown in Figure 6. The model grid 
was identical for the MYJ and Yonsei PBL 
schemes and is shown in Figure 6a, while the 
model grid was slightly different for the GFS PBL 
scheme and is shown in Figure 6b. 

The AMU plotted graphs of the observed 
steady-state wind speed at all three towers and 
overlaid the forecast steady-state wind speed from 
each of the model configurations for 42 wind 
cycling and 14 null case model runs. There are 
three graphs per tower per day that display the 
output from the different model configurations. 
These graphs included the following 
configurations: 

• LAPS ARW 

o Yonsei PBL scheme and MM5 similarity 
surface layer scheme 

o MYJ PBL scheme and Eta similarity 
surface layer scheme 

• LAPS NMM 

o GFS PBL scheme and GFS similarity 
surface layer scheme 


o MYJ PBL scheme and Eta similarity 
surface layer scheme 

• ADAS ARW 

o Yonsei PBL scheme and MM5 similarity 
surface layer scheme 

o MYJ PBL scheme and Eta similarity 
surface layer scheme 



Figure 6. Maps of EAFB showing the locations of the 
three concrete runway towers (red dots) and the model 
grid points (black dots) for the MYJ and Yonsei PBL 
schemes (a) and GFS PBL scheme (b). The grid point 
closest to each corresponding tower used in the 
analysis is shown within each green ellipse. 


An example of a cycling event at Tower 224 
on 7 June 2008 is shown in Figure 7. The event 
began at 0200 UTC and indicates a wave-like 
behavior in the wind speed time series that ended 
at 1300 UTC (denoted by the blue shaded box). 
During this time there were six oscillations in wind 
speed indicated by a change of 3 to 6 ms-1 (6 to 
12 kt) occurring approximately every 45 minutes. 
All of the model configurations correctly forecast 
the general trend of the wind speeds with the 
strongest occurring from 0000 to 0200 UTC then 
decreasing until 0800 UTC then increasing again 

Tower 224 7Jun08 LAPS ARW a 


Tower 224 MYJ Eta Yonsei MM5 



Tower 224 7Jun08 LAPS NMM 

Tower 224 —MYJ Eta GFS GFS 



Time (UTC) 


Tower 224 7Jun08 ADAS ARW 


Tower 224 MYJ Eta Yonsei MM5 



Figure 7. Graphs of a wind cycling case from Tower 
224 on 7 June 2008. In each graph the observed 
steady-state wind speed is shown by the dark red line 
and the forecast steady-state wind speed is shown by 
the orange and blue lines representing various model 
configurations as identified by the graph legends. The 
light blue shaded box indicates the duration of the wind 
cycling event. 



until the end of the cycling event. Only the ADAS 
ARW configurations showed any indication of the 
wind speed cycling within the general trend. The 
ADAS ARW MYJ Eta configuration indicated four 
to five oscillations whereas the ADAS ARW 
Yonsei MM5 configuration indicated three 
oscillations within the event timeline. The MYJ Eta 
configuration also had larger magnitudes in the 
wind speed oscillation compared to the Yonsei 
MM5. This result was consistent for the other two 
towers for this event. 

A review of graphs at Tower 224 from a null 
case on 9 June 2007 is shown in Figure 8. Wind 
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Figure 8. Graphs of a null case from Tower 224 on 9 
June 2007. In each graph the observed steady-state 
wind speed is shown by the dark red line and the 
forecast steady-state wind speed is shown by the 
orange and blue lines representing various model 
configurations as identified by the graph legends. 


cycling was not observed at any of the EAFB 
towers on this day. The model was run from 0600 
to 1800 UTC in the six configurations identical to 
the cycling events. During this time, the observed 
winds indicated a general weakening trend that all 
model configurations forecast. The LAPS NMM 
configurations indicated a low bias while the LAPS 
ARW and ADAS ARW indicated no overall bias 
and produced a better forecast than the LAPS 
NMM. 

4.2 Model Indications of Wind Cycling Events 

None of the model configurations were able to 
accurately forecast the timing and magnitude of 
the steady-state wind speed oscillations at model 
grid points nearest to the wind towers. However, 
the model can still provide valuable information to 
the forecasters to indicate whether a wind cycling 
event may occur. The AMU compared plots of 
forecast steady-state winds over a large part of the 
model domain for all of the wind cycling events 
and the null cases to see if the model could 
discern between the event days and null days. 

Figure 9 shows four time intervals of model 
forecast steady-state winds plotted in the EAFB 
area for a wind cycling event day, 7 June 2008, 
from the ADAS ARW MYJ Eta configuration. The 
figure shows a progression of changing wind 
speed and direction near the concrete runway 
beginning with westerly winds of 7-10 kt at 0300 
UTC (Figure 9a), then southwesterly at 10-15 kt at 
0400 UTC (Figure 9b), changing to south- 
southeast at 10-12 kt at 0500 UTC (Figure 9c) and 
finally southeast at 15-20 kt at 0600 UTC (Figure 
9d). As the northwesterly winds move across the 
Tehachapi Mountains and southern extent of the 
Sierra Nevada Mountains, they also move through 
the Tehachapi Pass that lies between the 
northeast end of the Tehachapi Mountains and the 
southern end of the Sierra Nevada Mountains (see 
Figure 1). During wind cycling events, the winds 
moving through the Tehachapi Pass will extend all 
the way to EAFB causing the wind speed and 
direction changes as the winds oscillate through 
the pass. The blue line in Figure 9 shows the 
approximate leading edge of the northwesterly 
winds moving across the mountains and through 
the pass. The winds moving through the pass 
cause the line to bulge eastward towards EAFB 
over time and disrupt the existing wind flow in the 
region. It is interesting to note that the Tehachapi 
Pass Wind Farm, located just east and south of 
the Tehachapi Pass, is one of California’s largest 
wind farms used for generating electricity (Edison 
International 2008). 



Figure 9. Plots of ADAS ARW MYJ Eta model forecast steady-state winds for 7 June 2008 in the EAFB area for 
four model output times of 0300 UTC (a), 0400 UTC (b), 0500 UTC (c) and 0600 UTC (d). The location of EAFB is 
indicated by the white circle. The approximate location of the concrete runway is shown by the red line. The blue line 
indicates the leading edge of westerly winds. 


Figure 10 shows four time intervals of model 
forecast steady-state winds plotted in the EAFB 
area for a null case, 9 June 2007 from the ADAS 
ARW MYJ Eta configuration. The figure shows a 
nearly constant southwest wind flow with speeds 
of 5-10 kt throughout the series of images from 
0700 to 1000 UTC near the concrete runway. The 
approximate leading edge of the northwesterly 
winds moving across the mountains and through 
the pass (blue line) was semi-stationary 
throughout the entire 12-hr model run. The 
northwesterly wind flow over the mountains was 
weaker in this null case compared to the cycling 
case shown in Figure 9, and the model correctly 
forecast a null event at EAFB. 

4.3 Subjective Analysis Summary 

Based on the subjective analysis of all wind 
cycling and null case days, the ARW runs 
outperformed the NMM runs. The NMM core 
consistently under-predicted wind speeds by the 
largest margin and did not capture changes in 


wind direction as well as the ARW core. Changing 
the model core seemed to have the biggest impact 
on the forecasts, while changing the model 
physics seemed to have the least impact on the 
forecasts. Although it appears the MYJ Eta 
scheme may have captured the cycling a bit better 
than the other configurations within the ARW core. 
The model did not forecast wind speeds better for 
the null days vs. the wind cycling days, but it 
clearly differentiated between cycling days and 
null cases used in this task. This would help the 
forecaster discern when a wind cycling case would 
occur. 

5. OBJECTIVE ANALYSIS 

For the objective analysis, observed wind 
speed was compared to forecast wind speed using 
the latest version of the Model Evaluation Tools 
(MET) software. This software was developed by 
the NCAR Developmental Testbed Center. It is a 
state-of-the-art suite of verification tools that uses 
output from the WRF model to compute standard 




Figure 10. Plots of ADAS ARW MYJ Eta model forecast steady-state winds for 9 June 2007 in the EAFB area for 
four model output times of 0700 UTC (a), 0800 UTC (b), 0900 UTC (c) and 1000 UTC (d). The location of EAFB is 
indicated by the white circle. The approximate location of the concrete runway is shown by the red line. The blue line 
indicates the leading edge of westerly winds moving across the Tehachapi Mountains. 


verification scores comparing gridded model data 
to point or gridded observations. Each statistic 
computed for this task compared the gridded WRF 
model data, available every 15 minutes, to the 
observations from the 12 wind towers at EAFB. In 
addition, the observed wind direction from the 12 
wind towers was compared to forecast wind 
direction from the WRF model data. However, the 
MET software does not currently support objective 
analysis of wind direction. Therefore, the forecast 
wind direction was manually pulled from the model 
data and all statistics were computed using 
Microsoft Excel. 


indicated wind cycling events were not separated 
from those that did not indicate wind cycling. Thus, 
the objective analysis is not a stand-alone analysis 
that indicates whether the wind cycling 
phenomena were captured, rather it only shows 
how well the model performed overall. The 
subjective analysis in Section 4 was used to 
determine if the model captured the wind cycling 
events. However, comparison of the objective 
analyses from the null cases to the wind cycling 
cases may help to determine whether the model is 
adept at forecasting a mesoscale process, such as 
wind cycling. 


Many output statistics are available within the 
MET software. This study looked at three of those 
statistics for wind speed: the forecast vs. the 
observed mean, the mean error, and the Pearson 
Correlation Coefficient (PCC); and two statistics 
for wind direction: the mean error and the PCC. 
The statistics compared all 12 towers combined to 
the corresponding locations in the model forecast 
output at a 15-minute interval. Towers that 


The mean forecast vs. observed wind speed 
comparison shows how well the forecast wind 
speed corresponded to the observed wind speed. 
The mean forecast and observed wind speed, / 

and o , respectively, were computed as follows: 
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where: 

n = 12 for the number of tower locations, 

f = forecast wind speed at a tower location, 
and 

o, = observed wind speed at a tower. 

The mean error, ME, is a measure of the 
overall bias of the wind speed or direction. A 
perfect forecast has ME = 0. It is defined as: 

ME -°,)=f-o (3) 


where n,f, and o, are defined as above and: 

/ = average forecast wind speed/direction at 
12 tower locations, and 

o = average observed wind speed/direction at 
1 2 towers. 

The PCC, r, measures the strength of the 
linear association between the forecast and 
observed wind speed or direction. It is defined as: 
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Where: 

ft = forecast wind speed/direction at all tower 
locations for one output period, 

Oj = observed wind speed/direction at all tower 
locations for one output period, 

/ = average forecast wind speed/direction for 
12-hour forecast, and 

o = average observed wind speed/direction 
for 12-hour forecast. 


The PCC can range between -1 and 1; 1 indicates 
a perfect correlation, -1 indicates a perfect 
negative correlation, and 0 indicates no correlation 
between the forecast and observations. 
Specifically, the PCC for wind speed measures 
whether large values of forecast wind speed tend 
to be associated with large values of observed 
wind speed (positive correlation), whether small 
values of forecast wind speed tend to be 
associated with large values of observed wind 
speed or vice versa (negative correlation), or 
whether values of both variables are unrelated 
(correlation near 0). For wind direction, the PCC 
measures the rotation of the wind with a perfect 
positive correlation indicating that the winds are 
shifting in the same direction with the same 
magnitude and a perfect negative correlation 
indicating that the winds are shifting in opposite 
directions with the same magnitude. 

5.1 Forecast vs. Observed Wind Speed 

All WRF model configurations under-predicted 
the wind speed throughout the forecasts for all 
wind cycling case days, except for the 30/31 July 
2008 run, which was the marginal wind cycling 
case day. Figure 11 shows the forecast vs. 
observed mean wind speed for three of those wind 
cycling case days: 22 December 2006 (Figure 
11a), 4/5 March 2008 (Figure 11b), and 30/31 July 
2008 (Figure 11c), and one null case: 9 June 2007 
(Figure lid). As can be seen from Figure 11a and 
b, the only model configuration that did not under- 
predict the wind speed was the LAPSARW_MYJ 
on 4/5 March (Figure 11b). For 30 July (Figure 
11c), all model configurations over-predicted the 
wind speed, except for the LAPSARW_Yonsei 
configuration. For the 9 June null case (Figure 
lid), the model slightly under-predicted the wind 
speed. 

5.2 Mean Error for Wind Speed and Direction 

As stated above, the mean error is a measure 
of the overall bias of the wind speed or direction. 
Figure 12 shows the mean error for wind speed for 
the same three wind cycling case days and one 
null case as in Figure 11. Again, it is apparent that 
all model configurations generally under-predicted 
the wind speed for both the 22 December (Figure 
12a) and 4/5 March (Figure 12b) cases. In both 
cases, the LAPSNMM_GFS was the worst 
performer, under-predicting the wind speed by 4 to 
8 m/s throughout the forecast. Figure 12a reveals 
that the only model configurations to over-predict 
wind speed during the forecast were both of the 
ADAS runs. By the second half of the forecast, all 
model configurations except the LAPSNMM_GFS 



Figure 1 1 . Chart of the forecast mean wind speed from the six WRF model configurations vs. the observed mean 
wind speed from the EAFB towers for the 12-hour forecast period on a) 22 December 2006, b) 4/5 March 2008, c) 30 
July 2008, and d) 9 June 2007. 


had nearly the same mean error values. Figure 
12b shows that all of the ARW runs were clustered 
throughout the entire 12-hour forecast, indicating 
that they captured and missed all of the same 
fluctuations in the observed wind speed. The 
mean error for 30/31 July (Figure 12c) shows the 
same range in mean error values as for 22 
December and 4/5 March, but reversed. That is, in 
general the model over-predicted the wind speeds 
throughout the forecast with the mean error falling 
in the positive range. Unlike the other two wind 
cycling cases, the NMM runs outperformed all 
ARW runs with the LAPSNMM_MYJ performing 
the best and the ARW MYJ runs performing the 
worst. For the null case (Figure 1 2d), the latter half 
of the forecast was more accurate than the 
beginning of the forecast. The mean error range 
for this case was smaller than for the wind cycling 
cases, but this was expected since the wind speed 
values were half of those for the wind cycling 
days. A comparison of the bias, given below, will 
give a more accurate comparison of the wind 
cycling days to the null cases. 


Table 4 lists the average mean error and bias for 
wind speed for the entire 12-hour forecast for each 
model configuration for each wind cycling case 
day and the two null cases. The smallest average 
mean error/bias is highlighted in red for each case 
day and the largest average mean error/bias is 
highlighted in blue. Overall, both the 
LAPSARW_MYJ and ADASARW_MYJ produced 
the smallest average mean errors and had the 
smallest bias of all configurations, followed by the 
ADASARW_Yonsei configuration. Clearly the 
worst performer among all model configurations 
was the LAPSNMM_GFS which was the worst in 
seven out of the nine cases. 

Figure 13 shows the mean error for wind 
direction for the same three wind cycling case 
days and one null case. In general, the model 
configurations that used the same cores but 
different physics give nearly the same mean error 
for all wind cycling and null cases. In fact, all 
model configurations seemed clustered throughout 
the forecast, indicating that they captured and 
missed all of the same shifts in wind direction. 
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Figure 12. Chart of the mean error for wind speed from the six WRF model configurations for the 12-hour forecast 
period for a) 22 December 2006, b) 4/5 March 2008, c) 30 July 2008, and d) 9 June 2007. 


Table 4. Average mean error and bias (parenthesis) for the 12-hour forecast for each model configuration for 
the wind cycling and null cases. The smallest average mean error/bias is highlighted in red for each case day. 


Model Configurations 

Date 

LAPSARW 

Yonsei 

LAPSARW 

MYJ 

ADASARW 

Yonsei 

ADASARW 

MYJ 

LAPSNMM 

GFS 

LAPSNMM 

MYJ 

22 Dec 06 

-1.96 (0.77) 

-2.28 (0.93) 

-1.72(1.03) 

-1.43 (1.09) 

-4.58 (0.53) 

-3.14(0.75) 

30 Jan 08 

-2.86 (0.79) 

-1.81 (0.88) 

-3.43 (0.74) 

-2.25 (0.84) 

-7.45 (0.39) 

-5.78 (0.54) 

14 Feb 08 

-1.73 (0.86) 

-0.79 (0.97) 

-2.62 (0.71) 

-2.7 (0.68) 

-5.1 (0.48) 

-3.25 (0.69) 

4/5 Mar 08 

-1.23 (0.82) 

-0.6 (0.89) 

-1.71 (0.75) 

-1.19(0.81) 

-4.96 (0.35) 

-3.77 (0.49) 

4 Jun 08 

-3.52 (0.77) 

-2.57 (0.84) 

-2.5 (0.83) 

-1.1 (0.95) 

-7.44 (0.43) 

-5.69 (0.58) 

7 Jun 08 

-1.17(0.88) 

-0.75 (0.92) 

-0.94 (0.93) 

-0.75 (0.94) 

-4.1 (0.54) 

-2.33 (0.75) 

30/31 Jul 08 

1.64(1.23) 

3.58(1.5) 

2.01 (1.28) 

3.21 (1.45) 

-1.68 (0.76) 

-0.2 (0.97) 

9 Jun 07 (null case) 

-0.58 (0.92) 

-0.99 (0.78) 

0.3 (0.99) 

-0.31 (0.82) 

-1.91 (0.47) 

-1.67 (0.53) 

1 Nov 07 (null case) 

-0.18(0.86) 

0.17(1.12) 

-0.5 (0.66) 

0.05 (1.05) 

-0.36 (0.77) 

-0.36 (0.77) 


Overall, no one model configuration was the best 
or worst performer for a majority of the cases. The 
discrepancies between model configurations for 
mean error of wind direction were smaller than for 
the wind speed, indicating that the model may 


have been better at forecasting wind direction than 
wind speed. In the highlighted cases, the smallest 
mean error occurred on 30/31 July (Figure 13c), 
while the largest mean error occurred for the null 
case on 9 June (Figure 13d). 
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Figure 13. Chart of the mean error for wind direction from the 6 WRF model configurations for the 12-hour 
forecast period for a) 22 December 2006, b) 4/5 March 2008, c) 30 July 2008, and d) 9 June 2007. 


5.3 Pearson Correlation Coefficient for Wind 

Speed and Direction 

The PCC can indicate whether the model 
configurations caught the overall trend of the wind 
speed. That is, it answers the question of whether 
the model winds fluctuated positively and 
negatively with the same magnitude as the 
observed winds. The closer the correlation 
coefficient is to 1 , the better the model was able to 
capture these trends. When comparing forecast 
vs. observed wind speed, only positive coefficients 
indicate any value in the model forecasts. 

Figure 14 shows the PCC for wind speed from 
the 12-hour forecasts of all wind cycling and null 
cases simulated with the six model configurations. 
The most obvious feature is that on both 30 
January 2008 and 4 June 2008 each model 
configuration did a poor job at capturing the trends 
in the observed winds. The best model forecast 
days were 4/5 March 2008 and 30/31 July 2008. 
Both days had correlation coefficients of 0.6 or 
above for all model configurations. The best 
performer for 4/5 March was the LAPSNMM_MYJ 


and the LAPSNMM_GFS for 30/31 July; however, 
the bias for these two configurations was poor 
compared to the other configurations for the same 
day. This indicates that although the model 
forecast wind speeds were too low, the model 
configuration was able to capture the fluctuations 
in wind speed maximums and minimums. 
Disregarding the PCCs for the two worst days (30 
January 2008 and 4 June 2008), the consistently 
good performers were the LAPSARW_Yonsei and 
LAPSNMM_MYJ configurations. 

For wind direction, the PCC can indicate 
whether the model configurations caught the 
overall shifts in wind direction. That is, it answers 
the question of whether the model wind direction 
fluctuated positively and negatively with the same 
magnitude as the observed wind direction. The 
closer the correlation coefficient is to 1 , the better 
able the model to capture the same magnitude 
and direction of the observed wind shift. A 
correlation coefficient closer to -1 indicates that 
the model was able to capture the magnitude of 
the wind shift, but in the opposite direction. 


Pearson Correlation Coefficient - Wind Speed 



Figure 14. Chart showing the Pearson Correlation Coefficient for wind speed for the 12-hour forecasts for all wind 
cycling and null cases for the six model configurations. 


Figure 15 shows the PCC of wind direction for 
the 12-hour forecasts for all wind cycling and null 
cases for the six model configurations. Unlike the 
PCC for the wind speed forecasts, each model 
configuration did a poor job at capturing any of the 
trends in the observed winds on 30/31 July 2008. 
In fact, the average PCC for this day for all model 
configurations combined is 0, or no correlation. 
The model configurations did well capturing the 
shifts in wind direction for the rest of the wind 
cycling case days. Overall, the model was better 
able to capture shifts in wind direction for the wind 
cycling days than for the null cases. The best 
model forecast day was 7 June 2008, which had a 
correlation coefficient of 0.6 or above for all model 
configurations. As was found with the mean error 
calculations, there is no indication of one model 
configuration as clearly the best or worst 
performer. 

5.4 Objective Analysis Summary 

Based on the objective analysis of all wind 
cycling and null case days, it appears that overall 


the ARW runs outperformed the NMM runs. 
Although the NMM core was more successful in 
capturing the increases and decreases in wind 
speed, it did a poor job at capturing the magnitude 
of the wind and consistently under-predicted wind 
speeds by the largest margin. Changing the model 
core seemed to have the biggest impact on the 
forecasts, while changing the model physics 
seemed to have the least impact on the forecasts. 
It does not appear the model does any better 
forecasting wind speeds for the null days vs. the 
wind cycling days. 

6. CONCLUSIONS 

Occasionally, the space shuttle must land at 
EAFB in Southern California when weather 
conditions at KSC violate Flight Rules. This can 
create a challenge for SMG as the complex terrain 
in and around EAFB makes forecasting surface 
winds difficult. In particular, “wind cycling" cases, 
in which the wind speeds and directions oscillate 
among towers near the EAFB runway, present a 
challenging forecast problem for shuttle landings. 


Pearson Correlation Coefficient - Wind Direction 



Figure 15. Chart showing the Pearson Correlation Coefficient for wind direction for the 12-hour forecasts for all 
wind cycling and null cases for the six model configurations. 


An accurate depiction of the winds along the 
runway is crucial in making the landing decision. 
Since global and national scale models cannot 
properly resolve the wind field due to their coarse 
horizontal resolutions a properly tuned high- 
resolution mesoscale model, like WRF, is needed. 

The evaluation described in this report was 
designed to assess the skill of different WRF 
model configurations for forecasting wind cycling 
events at EAFB. This study was done in two steps: 
1) Initializing both cores of the WRF model with 
either the LAPS or ADAS software, while also 
varying some of the physical parameters, and 2) 
Assessing the accuracy of model output by 
comparing surface wind forecasts to local EAFB 
tower observations. 

SMG identified seven wind cycling case days 
from December 2006 to July 2008. Six of the 
seven days were considered strong wind cycling 
cases, while one was a marginal case. The AMU 
also randomly chose two null cases. To compare 
the different model configurations, the WRF model 


was run for 12 hours at a 1-km horizontal grid 
spacing over EAFB and the surrounding 
mountainous areas using data from the identified 
case days. Data ingested by the model through 
the data analysis packages included Level II 
WSR-88D data, GOES visible and infrared 
satellite imagery, MADIS data, and local EAFB 
wind tower data. Each candidate day consisted of 
six runs using different model configurations, for a 
total of 42 runs for the wind cycling case days and 
14 for the null cases. The different configurations 
included: 

• LAPS-ARW with the Yonsei University PBL 
scheme and MM5 similarity surface layer 
scheme, 

• LAPS-ARW with the Mellor-Yamada-Janjic 
PBL scheme and ETA similarity surface 
layer scheme, 

• LAPS-NMM with the NCEP GFS PBL 
scheme and NCEP GFS surface layer 
scheme, 



• LAPS-NMM with the Mellor-Yamada-Janjic 
PBL scheme and ETA similarity surface 
layer scheme, 

• ADAS-ARW with the Yonsei University PBL 
scheme and MM5 similarity surface layer 
scheme, and 

• ADAS-ARW with the Mellor-Yamada-Janjic 
PBL scheme and ETA similarity surface 
layer scheme. 

6.1 Results 

The major results from the evaluation show 
the following: 

• Subjective analysis, 

o Overall, the ARW runs outperformed the 
NMM runs. 

o Changing the model core seemed to 
have the biggest impact on the 
forecasts, while changing the model 
physics seemed to have the least 
impact on the forecasts. 

o Model configurations that used the MYJ 
PBL scheme seemed to slightly 
outperform other PBL schemes. 

o The model was able to differentiate 
between wind cycling days and null 
cases. 

• Objective analysis, 

o Overall the ARW runs outperformed the 
NMM runs. 

o The NMM core was more successful in 
matching the increasing and decreasing 
trends in wind speed with the 
observations, but did a poor job at 
capturing the magnitude of the wind and 
consistently under-predicted wind 
speeds by the largest margin. 

o As was found in the subjective analysis, 
changing the model core seemed to 
have the biggest impact on the 
forecasts, while changing the model 
physics seemed to have the least 
impact on the forecasts. 


In both the subjective and objective analyses, 
the AMU found the ARW core and MYJ PBL 
scheme performed better than the other model 
configurations. The model did not produce a better 
wind forecast on null days vs. wind cycling days. 
The model was able to differentiate between wind 
cycling days and null cases which would provide 
added value to the shuttle landing forecast. 

6.2 Recommendations 

Of the six model configurations tested, the 
AMU recommends either the ADAS ARW MYJ or 
LAPS ARW MYJ configuration for operational use 
for predicting wind cycling events at EAFB for the 
following reasons: 

• Both configurations consistently had the 
lowest bias for wind speed compared to the 
other model configurations, 

• Both configurations best captured wind 
speed oscillations when compared to the 
observed wind speeds at the concrete 
runway towers, and 

• The NMM core produced wind speed 
forecasts well below those observed and 
did not capture the wind direction changes 
well on the cycling days. 

6.3 Future Work 

The goal of this work was to assess different 
WRF model options and to determine how well the 
model could predict surface wind speed and 
direction at EAFB and if one model configuration 
performed better than others. While that goal was 
met, the data indicated further investigations 
outside the scope of this task could include 
sophisticated signal analysis, rotating the model 
coordinate systems orthogonal and/or parallel to 
the prevailing wind flow and closely comparing the 
observed and forecast wind speed cycles to wind 
direction cycles which may lead to discoveries of 
why the winds cycle as they do and how that 
relates to model configuration and performance. 


o It does not appear that the model did 
any better forecasting wind speeds for 
the null days vs. the wind cycling days. 

o Model configurations that used the MYJ 
PBL scheme seemed to slightly 
outperform other PBL schemes. 
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